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Real Party in Interest 



The present application is assigned to Minolta, Co., Ltd., a Japanese corporation. 



Related Appeals and Interferences 



There are no other prior or pending appeals, interferences or judicial proceedings 
known to Appellant, Appellant's legal representative, or the assignee which may be related 
to, directly affect or be directly affected by, or have a bearing on the Board's decision in 
this appeal. 

III. Status of Claims 

The application contains 19 claims, all of which are currently pending. Claims 9-11 

have been allowed, and claim 16 has been identified as containing allowable subject matter. 
All of the remaining claims, namely claims 1-8, 12-15 and 17-19 stand finally rejected, and 
form the basis for this appeal. 

IV. Status of Amendments 

There were no amendments filed subsequent to the final Office Action. 
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V. Summary of Claimed Subject Matter 

The claimed invention is directed to methods and devices for processing digital 
image data. Figure 1 illustrates an example of the type of environment in which the 
claimed invention can be utilized. A digital camera 1 records an image of an original 2, 
such as a page of a book or magazine. (Page 11, lines 25-27). Once the digital image has 
been captured by the camera, the image data is subjected to various types of processing, to 
remove noise from the image and to compress the image data. (Page 1, lines 17-20). 
Different types of processing are performed on different types of image data. For instance, 
photographic data is processed in a manner different from textual data. (Page 1, lines 20- 
21; page 1, line 30 to page 2, line 13). 

Accordingly, it is desirable to be able to delineate the different areas of an image 
that require different respective types of processing. For example, Figure 37 illustrates an 
image that contains photographic information 242 and 246, graphical data 244, and textual 
information 248. These different areas are separately identified, and enclosed by 
rectangular boxes defined with horizontal and vertical lines. (Page 2, line 14 to page 3, 
line 5). 

The ability to delineate the different portions of an image can be adversely affected 
by distortion of the captured image due to the relative orientation of the camera 1 and the 
original object 2. The claimed invention is concerned with two different types of 
distortion. One type of distortion is referred to herein as the swing of the camera 1, and is 
explained with reference to Figure 13. Referring thereto, the optical axis of the camera 
lens is represented by the line 151. The original object is assumed to be aligned with the x 
and y axes depicted in the figure. When the optical axis 151 of the camera lens is parallel 
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the z axis, no swing distortion occurs. However, if the camera is rotated about either or 
both of the x and y axes, as represented by the angles y and a, swing distortion results. 
(Page 23, line 23 to page 24, line 9). The effect of swing distortion is to cause rectangular 
originals to appear to have a trapezoidal form within the image. (Page 3, lines 6-11). An 
example of this phenomenon is shown in Figure 3 1 . 

The second type of distortion is known as skew. This occurs when the digital 
camera is rotated about its optical axis, as represented by the angle p in Figure 13. (Page 
24, lines 10-15). When this occurs, the original is rotationally shifted relative to the overall 
image, as depicted in Figure 39. This type of distortion complicates the ability to optimally 
process the different portions of the image, as described in the specification at page 4, lines 
3-21, with reference to Figure 40. 

In one embodiment of the invention, the skew distortion is corrected by an 
inclination correction unit 1 10, depicted in the block diagram of Figure 3. The procedure 
carried out by this unit to correct the image data is represented in the flowchart of Figure 5. 
As a first step, S22, the image received and captured by the camera can be reduced, for 
example by an averaging process. This reduction simplifies the ability to identify textual 
portions of the image. (Page 14, line 20 to page 15, line 21). Thereafter, the received 
image is rotated by an angle 9 (S23). Two-dimensional differentiation is performed on the 
rotated image data (S25), and horizontal and vertical projection histograms are computed 
(S26 and S27), as depicted in Figure 9. (Page 15, line 22 to page 17, line 10). The 
distribution of the projection histograms is then calculated (S28), to derive a value that 
characterizes the image at the rotational angle 0. (Page 17, lines 26-32). 
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The angle 0 is then incremented by a predetermined amount a, at step S30, and the 
foregoing process is repeated for a number of different rotation angles. (Page 17, line 33 
to page 18, line 9). 

As illustrated in Figure 9, when the image is not aligned with the vertical and 
horizontal axes, the histograms tend to be relatively smooth, and therefore have generally 
lower distribution values. Conversely, when the image is properly oriented relative to the 
horizontal and vertical directions, as shown in Figure 10, the peaks in the histogram are 
much greater, and therefore the histograms exhibit a higher distribution value. At step S31, 
the rotation angle 9 that resulted in the highest distribution value is chosen as the proper 
orientation of the original, and the image is rotated to this angle at step S32. (Page 17, 
lines 11-26; page 18, lines 10-20; page 18, line 30 to page 19, line 18). 

In a second embodiment of the invention, correction of the skew angle is combined 
with correction due to swing of the camera. An example of this embodiment is illustrated 
in Figure 12. As depicted therein, the digital camera includes a horizontal swing angle 
calculation unit 132, a vertical swing angle calculation unit 136, and a skew rotation angle 
calculation unit 134. The calculations performed by each of these units are provided to a 
geometric transformation unit 138 to correct distortion due to swing and skew. (Page 22, 
lines 18-33; page 27, line 29 to page 30, line 4). 
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VI. Grounds of Rejection to be Reviewed 



The final Office Action presents the following four grounds of rejection for review 
on this appeal: 

1. The rejection of claims 1-3, 6 and 17-19 under 35 U.S.C. § 102, as being 
anticipated by the Cullen et al. patent (US 5,452,374); 

2. The rejection of claims 4, 5, 7 and 8 under 35 U.S.C. § 103, as being 
unpatentable over the Cullen et al. patent in view of the Mackinnon et al. 
patent (US 6,148,115); 

3. The rejection of claims 12-14 under 35 U.S.C. § 102, as being anticipated 
by the Morimura patent (US 5,940,128); and 

4. The rejection of claim 15 under 35 U.S.C. § 103 as being unpatentable over 
the Morimura patent. 
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VII. Argument 

A. Claims 1-3 and 17-19. 

Claims 1-3, 6 and 17 stand finally rejected as being anticipated by the Cullen patent. 
The Cullen patent discloses a rectangle construction technique to determine the skew angle 
of an input image. In general, the procedure disclosed in the Cullen patent attempts to 
define a boundary rectangle for each word or sentence in a document. See, for example, 
column 5, lines 4-12. The rectangle construction technique utilizes black and white pixel 
data in each scan line of the image. Referring to Figure 3, four consecutive scan lines 300- 
303 are vertically compressed into a single scan line, e.g. 312, and each byte of the 
compressed scan line is then horizontally compressed to identify either a black byte or a 
white byte, as illustrated at 315 and 316. 

Thereafter, each compressed scan line is processed for run length extraction, which 
defines sets of contiguous black pixels in the compressed scan line. Figure 4 illustrates an 
example of this run length extraction. Referring to Figure 5, successive compressed scan 
lines are then evaluated to construct rectangles. Basically, each run length of black bits 
defines a rectangle, and when the run lengths of two successive compressed scan lines 
overlap one another, they are added to a common rectangle. In essence, therefore, the 
technique of the Cullen patent defines rectangles by identifying areas of the image in which 
black pixels are grouped together, as in the case of a word. 

Once all of the rectangles have been defined, a skew angle is determined by 
measuring the offset between associated rectangles. Figure 8a illustrates an example in 
which the rectangle 802 is offset from the rectangle 801 by an amount that defines a skew 
angle 805. 
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Claim 1 recites an image processing device having, among other elements, an edge 
detection unit to detect edge data in image data, and a rotation unit "to rotate said detected 
edge data." Referring to Fig. 7 of the application, for example, the edge data in the image 
is detected, and this edge data is then rotated, as indicated by the counter clockwise arrows. 
Claim 1 further recites an operation unit to derive a characteristic amount "of said rotated 
edge data." Referring to Figs. 9 and 10, this characteristic amount is represented by the 
distribution of the horizontal and vertical projection histograms 227 and 229. Finally, 
claim 1 recites a unit to detect the inclination angle of the original image data "based on 
said derived characteristic amount. " 

In summary, therefore claim 1 recites an image processing device in which edge 
data is first detected, this detected edge data is then rotated, and a characteristic amount is 
derived from the rotated edge data. Based upon this characteristic amount, the inclination 
angle of the image is determined. Claim 17 recites the corresponding method steps. 

In rejecting claim 1, the final Office Action states that the Cullen patent discloses an 
edge detection unit 204 (Fig. 2c), a rotation unit 207, an operation unit 603 to derive a 
characteristic amount, and an inclination unit 604 to detect the inclination angle of the 
image data. However, even if the Cullen patent can be interpreted to disclose each of these 
individual elements, they do not cooperate with each other in the same manner as recited in 
claim 1 to produce the claimed result. As shown in Fig. 2c of the Cullen patent, the 
rotation of the edge data, which is identified as step 207 in the Office Action, occurs after 
step 206, which corresponds to the flow chart of Fig. 6 (see column 2, lines 54-56). 
Consequently, step 603 of Fig. 6, which was identified in the Office Action as 
corresponding to the derivation of a characteristic amount, occurs before the rotation of 
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step 207. Thus, the Cullen patent does not teach that the characteristic amount determined 
in step 603 is based upon rotated edge data. Rather, in contrast to the subject matter 
recited in claim 1 , the Cullen patent discloses a technique in which the inclination angle is 
first detected, and the rotation only occurs after such a determination has been made. 

In summary, the claims recite a procedure in which the received image is first 
rotated to a predetermined angle, and a characteristic amount is then determined for that 
angle. On the other hand, the Cullen patent first determines a characteristic amount that 
defines the skew angle, and then rotates to compensate for the skew. This difference 
becomes more apparent when the rejection of claim 3 is considered. Claim 3 recites that 
the image processing device further comprises an inclination correction unit to correct the 
inclination of the received original image "based on said detected inclination angle." Thus, 
once the inclination angle has been detected in accordance with claim 1 , it is then corrected 
with the structure recited in claim 3. In rejecting claim 3, the final Office Action refers to 
item 207 of the Cullen patent, the skew correction step. This same step was identified in 
the rejection of claim 1 as corresponding to the claimed rotation unit. However, as can be 
seen from a reading of the claims, the rotation unit of claim 1 performs its function before 
the inclination angle is detected (since such detection is based upon rotated image data), 
whereas the correction unit of claim 3 operates after the inclination angle is detected. 
There is no disclosure in the Cullen patent that the skew correction step 207 is carried out 
both before and after the skew angle is calculated in step 206. 

As a result, the reference cannot be deemed to anticipate the subject matter of claim 
1, since it does not disclose an operation unit to derive a characteristic amount "of said 
rotated edge data, " and determination of the inclination angle on the basis of such a 
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characteristic amount. For similar reasons, it does not anticipate the method of claim 17, 
which recites the step of deriving a characteristic amount "of said rotated edge data", and 
correcting inclination based on that derived characteristic amount. Accordingly, it is 
respectfully submitted that claims 1-8 and 17-19 are patentable over the teachings of the 
Cullen reference, whether considered by itself or in combination with the McKinnon patent. 

B. Claim 6 

Claim 6 depends from claim 1 and recites that the operation unit which derives the 
characteristic amount forms a histogram of the edge data, and that the characteristic amount 
is the distribution of the histogram. The rejection of the claim refers to the histogram 
depicted in Figure 7 of the Cullen patent. However, there is no disclosure in the patent that 
the distribution of such a histogram is employed to characterize image data. Rather, the 
histogram is used to locate the left or right edge for a column of text. See column 14, lines 
15-31. In this process, only the highest point 704 of the histogram is of interest. The 
distribution of the other points is not considered. 

For additional reason, therefore, the subject matter of claim 6 is not anticipated by 
the Cullen patent. 

C. Claim 7 

Claim 7 depends from claim 6 and recites that a histogram is formed for each of the 
vertical and horizontal directions in the image. The Cullen patent only discloses that the 
histogram of Figure 7 is calculated for one direction, along the x axis. To this end, 
therefore, the final Office Action refers to Figure 4 of the Mackinnon patent. However, 
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this figure has nothing to do with the generation of histograms to characterize image data. 
Rather, it relates to the arrangement of a memory 226 that stores edge data. See column 
10, lines 48-67. It is not apparent how this teaching would lead a person of ordinary skill 
to utilize both vertical and horizontal histograms in the system of the Cullen patent. 

For at least this reason, the subject matter of claim 7 is not suggested by the Cullen 
and Mackinnon patents, whether considered individually or in combination. 

D. Claims 12-14. 

Claim 12-14 were rejected under 35 U.S.C. § 102, on the grounds that they were 
considered to be anticipated by the Morimura patent. Claim 12 recites an image processing 
device having an image receiving unit and two correction units. One of these correction 
units is a swing correction unit, to correct distortion caused by swing of the camera with 
respect to an original image, and the other is a skew correction unit, to correct distortion 
caused by skew of the camera with respect to the original image. In relevant part, the 
Office Action alleges that the Morimura patent teaches a swing correction unit, with 
reference to Fig. 16. 

The flow chart depicted in Figure 16 of the Morimura patent has nothing to do with 
the correction of distortion due to the swing of a camera. Rather, Figure 16 illustrates an 
operation for controlling a zoom lens, so as to provide an optimum enlargement ratio. 
Referring to Figure 17 of the patent, the objective of the process disclosed in Figure 16 is 
to maximize the amount of the image that is filled by the document la, and minimize the 
amount of background area 1 that appears in the image. This objective is achieved by 
setting the zoom lens of the camera at an appropriate magnification ratio. This setting does 
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not correct for, nor otherwise have anything to do with, image distortion, particularly that 
caused by camera swing. Rather, its function is to control the size of the object, e.g. 
document, within the captured image. 

In responding to Appellant's previous arguments along these lines, the final Office 
Action states that the claim does not identify the difference between the swing of the 
camera and the skew of the camera, and that "the skew of the camera and the swing of the 
camera would mean the same thing." The Office Action does not provide any support for 
this statement. As recognized in the prior art, such as the Cullen patent, skew refers to the 
situation in which horizontal and/or vertical components of an object in the image do not 
align with the frame of reference for the image. See, for instance, the Cullen patent at 
column 1, lines 32-35. A person of ordinary skill in the art would not interpret the term 
"swing" to cover this type of situation. Rather, as described in the application, the term 
"swing" is used in the context of the present invention in accordance with its ordinary and 
accustomed meaning, to connote movement about an axis, through an arc. 

No such movement is considered in the operation depicted in Fig. 16 of the 
Morimura patent. The only movement that takes place is the linear translation of the lens 
along the optical axis. Of particular significance, the zooming of the lens does not function 
to correct any type of distortion in the image. The sole function of the zooming is to obtain 
the optimum magnification ratio, so that the image of the document fills the viewing area as 
much as possible. 

In summary, therefore, while the Morimura patent discloses a skew correction unit, 
it does not contain any disclosure relating to swing of the camera, nor correction of 
distortion caused by such swing. If anything, the patent suggests that the camera 3 is 
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fixedly mounted on the post 2 (see Figs. 1, 3 and 4, for example) and therefore is not 
capable of swinging relative to the document being imaged. In any event, there is no 
disclosure suggesting that the operation depicted in Fig. 16 corrects any type of distortion 
in the image. 

For the foregoing reasons, claims 12-14 are not anticipated by the Morimura patent. 

E. Claim 15 

Claim 15 depends from claim 12 and recites an image reduction unit to reduce 
received original image data. The final Office Action appears to acknowledge that the 
Morimura patent does not disclose this structure, but it is not clear from the action what the 
basis is for rejecting the claim despite the absence of such a teaching. The rejection seems 
to assume that it would be obvious to utilize reduced image data in the system of the 
Morimura patent, but does not identify a reference or other support for this position. As 
such, a prima facie case of obviousness has not been established. 

F. Claims 14 and 15 

Claim 15 further recites an inclination unit to detect inclination of each of a plurality 
of portions of the reduced image data, and that both the skew correction unit and the swing 
correction unit correct distortion based on these detected inclinations. Claim 14 also recites 
this subject matter. The Morimura patent discloses that a differential computation is carried 
out to identify the external border of an original, and a Hough transformation is performed 
to correct for inclination, or skew, of the original. See, for example, column 5, lines 13- 
28. There is no disclosure, however, that the inclination of "each of a plurality of 
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portions" of the image is detected. Rather, the Morimura patent only detects the inclination 
of the document as a whole. 

Furthermore, there is no disclosure that the detected inclination is used to correct 
for both skew and swing distortions. If, in accordance with the rejection of claim 12, the 
flow chart of Figure 16 is considered to represent a "swing correction unit," there is no 
teaching that an inclination determined in the embodiments of Figures 7 or 1 1 , to correct 
skew, is employed in the procedure of Figure 16. As best as can be determined from the 
Morimura patent, the operation of Figure 16 is carried out independently of any inclination 
calculations that are performed. 

For these additional reasons, therefore, the subject matter of claims 14 and 15 is 
neither anticipated by, or obvious in view of, the Morimura patent. 
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VIII. Conclusion 



As can be seen, the references relied upon in the various grounds of rejection do not 
disclose the subject matter recited in the claims against which they have been applied. The 
rejections are not properly founded in the statute, and should be reversed. 



Respectfully submitted, 

Burns, Doane, Swecker&Mathis, L.L.P. 



Date: September 29, 2004 By: 




Registration No. 28,632 

P.O. Box 1404 

Alexandria, Virginia 22313-1404 
(703) 836-6620 
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Appendix A - The Pending Claims 

1. An image processing device comprising: 

an image receiving unit to receive an original image data; 

an edge detection unit to detect an edge data included in an image data related to 
said received original image data; 

a rotation unit to rotate said detected edge data; 

an operation unit to derive a characteristic amount of said rotated edge data; and 
an inclination detection unit to detect an inclination angle of said received original 
image data based on said derived characteristic amount. 

2. The image processing device according to claim 1 further comprising: 
a reduction unit to reduce said received original image data; wherein 

said edge detection unit detects an edge data included in said reduced image data. 

3. The image processing device according to claim 1 further comprising: 

an inclination correction unit to correct the inclination of said received original 
image data based on said detected inclination angle. 

4. The image processing device according to claim 1 wherein 

said edge detection unit detects said edge data by differentiating said reduced image 
data with a filter. 
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5. The image processing device according to claim 4 wherein 

said edge detection unit uses different filters for finding an edge data in a horizontal 
direction and for finding an edge data in a vertical direction. 

6. The image processing device according to claim 1 wherein 

said operation unit forms a histogram by projecting said detected edge data in a 
predetermined direction, and 

said characteristic amount is a distribution of said formed histogram. 

7. The image processing device according to claim 6 wherein 

said predetermined direction includes a vertical direction and a horizontal direction 
of said original image data. 

8. The image processing device according to claim 2 further comprising: a camera 
to pick up an image of an original; wherein 

the image data received by said image receiving unit is an image picked up by said 
camera. 

9. An image processing device comprising: 

an image receiving unit to receive an original image data; 

a process image forming unit to form an image data to be processed based on said 
received original image data; 

a geometric transformation unit to perform a swing rotation transformation on said 

formed image data using an angle as a parameter; 
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a first inclination detection unit to detect a first inclination of a first area in said 
formed image data; 

a second inclination detection unit to detect a second inclination of a second area 
opposing said first area in said formed image data; and 

a swing correction unit to perform a swing rotation transformation on said received 
original image data based on said first inclination and said second inclination. 

10. The image processing device according to claim 9 further comprising: 

a comparison unit to compare said first inclination and said second inclination; 
wherein 

said swing correction unit performs said swing rotation formation based on the 
comparison by said comparison unit. 

11. The image processing device according to claim 9 further comprising: 

a third inclination detection unit to detect an inclination of a left side area in said 
formed image data; 

a fourth inclination detection unit to detect an inclination of a right side area in said 

formed image data; wherein 

said first area is an upper side area in said formed image data, 

said second area is a lower side area in said formed image data, 

said processing image forming unit forms a rectangular formed by straight lines with 

detected inclinations of said upper side area, said lower side area, said left side area and 

said right side area, respectively, 
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said geometric transformation unit transforms coordinates of apexes of said formed 
rectangular image. 

12. An image processing device comprising: 

an image receiving unit to receive an original image data obtained with a camera 
picking up an original image; 

a swing correction unit to correct distortion of an image caused by a swing of said 
camera with respect to the original image; and 

a skew correction unit to correct distortion of an image caused by a skew of said 
camera with respect to the original image. 

13. The image processing device according to claim 12 wherein 

said skew correction unit corrects the distortion of the image corrected by said 
swing correction unit. 

14. The image processing device according to claim 12 further comprising: 

an inclination detection unit to detect an inclination of each of a plurality of portions 
in said received original image data wherein 

said swing correction unit and said skew correction unit correct distortion of said 
received original image data based on said detected inclinations. 

15. The image processing device according to claim 12 further comprising: 
an image reduction unit to reduce said received original image data; and 
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an inclination detection unit to detect an inclination of each of a plurality of portions 
of said reduced image data; wherein 

said swing correction unit and said skew correction unit correct a distortion of said 
received original image data based on said detected inclinations. 

16. The image processing device according to claim 12 further comprising: 
a process image forming unit to form an image to be processed based on said 

received original image data; 

a first swing angle detection unit to detect a swing angle in a first direction of said 

received original image data by performing a geometric transformation on said image to be 

processed; 

a skew angle detection unit to detect a skew angle of said received original image 
data by performing the geometric transformation on the image subjected to the geometric 
transformation by said first swing angle detection unit; and 

a second swing angle detection unit to detect a swing angle in a second direction of 
said received original image data by performing the geometric transformation on the image 
subjected to the geometric transformation by said skew angle detection unit; wherein 

said swing correction unit corrects said received original image data based on the 
detected swing angle in said first direction and the detected swing angle in said second 
direction; 

said skew correction unit corrects said received original image data based on said 
detected skew angle. 
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17. An image processing method comprising the steps of: 
receiving an original image data; 

detecting an edge data included in an image data to be processed related to said 
received original image data; 

rotating said detected edge data; 

deriving a characteristic amount of said rotated edge data; and 
correcting an inclination of said received original image data based on said derived 
characteristic amount. 

18. The image processing method according to claim 17, 

said image data to be processed is said received original image data. 

19. The image processing method according to claim 17, further comprising the 

step of; 

reducing said received original image data; wherein said image data to be processed 
is said reduced original image data. 
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Appendix B - Evidence 
(None) 
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Appendix C - Related Proceedings 
(None) 
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